Image processing system, processing method, and storage medium

ABSTRACT

An image processing system includes: an obtaining unit configured to obtain a tomographic image of an eye to be examined; an analysis unit configured to execute analysis required to obtain information indicating a degree of curvature of a retina from the tomographic image of the eye to be examined according to an imaging mode upon capturing an image of the eye to be examined; and a display control unit configured to display three-dimensional shape data of a retinal layer generated to obtain the information on a display device.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an image processing system, processingmethod, and storage medium.

2. Description of the Related Art

A tomography apparatus using an OCT (Optical Coherence Tomography),which utilizes interference caused by low coherent light, is known. Bycapturing an image of a fundus by such tomography apparatus, the stateof interior of retinal layers can be three-dimensionally observed.

Imaging by the tomography apparatus is receiving a lot of attentionsince it is a technique helpful to give more adequate diagnoses ofdiseases. As a mode of such OCT, for example, a TD-OCT (Time Domain OCT)as a combination of a broadband light source and Michelsoninterferometer is known. The TD-OCT measures interfering light withbackscattered light of a signal arm by scanning a delay of a referencearm, thus obtaining depth resolution information.

However, it is difficult for the TD-OCT to obtain an image fast. Forthis reason, as a method of obtaining an image faster, an SD-OCT(Spectral Domain OCT) which obtains an interferogram by a spectroscopeusing a broadband light source is known. Also, an SS-OCT (Swept SourceOCT) which measures spectral interference by a single-channelphotodetector using a fast wavelength swept light source is known.

In this case, if a shape change of a retina can be measured in atomographic image captured by each of these OCTs, a degree of progressof a disease such as glaucoma and a degree of recovery after treatmentcan be quantitatively diagnosed. In association with such technique,Japanese Patent Laid-Open No. 2008-073099 discloses a technique fordetecting boundaries of respective layers of a retina from a tomographicimage and measuring thicknesses of the layers based on the detectionresult using a computer, so as to quantitatively measure the shapechange of the retina.

With the technique of Japanese Patent Laid-Open No. 2008-073099described above, a tomographic image within a range designated on atwo-dimensional image is obtained, and layer boundaries are detected tocalculate layer thicknesses. However, three-dimensional shape analysisof retinal layers is not applied, and the shape analysis result is noteffectively displayed.

SUMMARY OF THE INVENTION

The present invention has been made in consideration of theaforementioned problems, and provides a technique which presentsthree-dimensional shape data of a retinal layer detected from atomographic image of an eye to be examined.

According to one aspect of the present invention, there is provided animage processing system comprising: an obtaining unit configured toobtain a tomographic image of an eye to be examined; an analysis unitconfigured to execute analysis required to obtain information indicatinga degree of curvature of a retina from the tomographic image of the eyeto be examined according to an imaging mode upon capturing an image ofthe eye to be examined; and a display control unit configured to displaythree-dimensional shape data of a retinal layer generated to obtain theinformation on a display device.

According to the present invention, three-dimensional shape data of aretinal layer detected from a tomographic image of an eye to be examinedcan be presented.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments (with reference to theattached drawings).

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing an example of the arrangement of animage processing system 10 according to one embodiment of the presentinvention;

FIG. 2 is a view showing an example of a tomographic image capturingscreen 60;

FIGS. 3A and 3B are flowcharts showing an example of the sequence ofprocessing of an image processing apparatus 30 shown in FIG. 1;

FIG. 4 is a view for explaining an overview of three-dimensional shapeanalysis processing;

FIGS. 5A to 5C are views for explaining an overview of three-dimensionalshape analysis processing;

FIG. 6 is a view showing an example of a tomographic image observationscreen 80;

FIGS. 7A to 7C are views showing an example of respective components ofthe tomographic image observation screen 80;

FIGS. 8A to 8D are views showing an example of respective components ofthe tomographic image observation screen 80;

FIGS. 9A and 9B are views showing an example of respective components ofthe tomographic image observation screen 80;

FIG. 10 is a view showing an example of the tomographic imageobservation screen 80;

FIG. 11 is a view showing an example of respective components of thetomographic image observation screen 80;

FIG. 12 is a view showing an example of the tomographic imageobservation screen 80;

FIG. 13 is a view showing an example of a tomographic image observationscreen 400;

FIG. 14 is a view showing an example of respective components of thetomographic image observation screen 400;

FIG. 15 is a view showing an example of the tomographic imageobservation screen 400; and

FIG. 16 is a view showing an example of respective components of thetomographic image observation screen 400.

DESCRIPTION OF THE EMBODIMENTS

Embodiments according to the present invention will be described indetail hereinafter with reference to the accompanying drawings.

First Embodiment

FIG. 1 is a block diagram showing an example of the arrangement of animage processing system 10 according to an embodiment of the presentinvention.

The image processing system 10 includes an image processing apparatus30, tomography apparatus 20, fundus image capturing device 51, externalstorage device 52, display device 53, and input device 54.

The tomography apparatus 20 is implemented by, for example, an SD-OCT orSS-OCT, and captures a tomography image indicating a three-dimensionalshape of a fundus using an OCT using interference caused by low coherentlight. The tomography apparatus 20 includes a galvanometer mirror 21,driving control unit 22, parameter setting unit 23, vision fixation lamp24, and coherence gate stage 25.

The galvanometer mirror 21 has a function of two-dimensionally scanningmeasurement light (irradiation light) on a fundus, and defines animaging range of a fundus by the tomography apparatus 20. Thegalvanometer mirror 21 includes, for example, two mirrors, that is, X-and Y-scan mirrors, and scans measurement light on a plane orthogonal toan optical axis with respect to a fundus of an eye to be examined.

The driving control unit 22 controls a driving (scanning) range andspeed of the galvanometer mirror 21. Thus, an imaging range in a planardirection (a direction orthogonal to an optical axis direction ofmeasurement light) and the number of scan lines (scanning speed in theplanar direction) on a fundus are defined.

The parameter setting unit 23 sets various parameters used in drivingcontrol of the galvanometer mirror 21 by the driving control unit 22.These parameters decide imaging conditions of a tomographic image by thetomography apparatus 20. For example, scan positions of scan lines, thenumber of scan lines, the number of images to be captured, and the likeare decided. In addition, a position of the vision fixation lamp,scanning range, and scanning pattern, coherence gate position, and thelike are also set. Note that the parameters are set based on aninstruction from the image processing apparatus 30.

The vision fixation lamp 24 suppresses movement of a viewpoint byplacing a bright spot in a visual field so as to prevent an eyeballmotion during imaging of a tomographic image. The vision fixation lamp24 includes an indication unit 24 a and lens 24 b. The indication unit24 a is realized by disposing a plurality of light-emitting diodes (LDs)in a matrix. Lighting positions of the light-emitting diodes are changedin correspondence with a portion as an imaging target under the controlof the driving control unit 22. Light from the indication unit 24 a isguided to the eye to be examined through the lens 24 b. Light emergingfrom the indication unit 24 a has a wavelength of, for example, 520 nm,and a desired pattern is indicated (lighted) under the control of thedriving control unit 22.

The coherence gate stage 25 is arranged to cope with, for example, adifferent ophthalmic axis length of an eye to be examined. Morespecifically, an optical path length of reference light (to beinterfered with measurement light) is controlled to adjust an imagingposition along a depth direction (optical axis direction) of a fundus.Thus, optical path lengths of reference light and measurement light canbe matched even for an eye to be examined having a different ophthalmicaxis length. Note that the coherence gate stage 25 is controlled by thedriving control unit 22.

In this case, a coherence gate indicates a position where opticaldistances of measurement light and reference light are equal to eachother in the tomography apparatus 20. By controlling the coherence gateposition, imaging on the side of retinal layers or that of an EDI(Enhanced Depth Imaging) method on the side deeper than the retinallayers is switched. When imaging is done using the EDI method, thecoherence gate position is set on the side deeper than the retinallayers. Hence, when an image of the retinal layers is captured beyond anupper portion side of a tomographic image, the retinal layers can beprevented from appearing in the tomographic image while being foldedback.

The fundus image capturing device 51 is implemented by, for example, afundus camera, SLO (Scanning Laser Ophthalmoscope), or the like, andcaptures a (two-dimensional) fundus image of a fundus.

The external storage device 52 is implemented by, for example, an HDD(Hard Disk Drive) or the like, and stores various data. The externalstorage device 52 holds captured image data, imaging parameters, imageanalysis parameters, and parameters set by an operator in associationwith information (a patient name, age, gender, etc.) related to an eyeto be examined.

The input device 54 is implemented by, for example, a mouse, keyboard,touch operation screen, and the like, and allows an operator to inputvarious instructions. For example, the operator inputs variousinstructions, settings, and the like for the image processing apparatus30, tomography apparatus 20, and fundus image capturing device 51 viathe input device 54. The display device 53 is implemented by, forexample, a liquid crystal display or the like, and displays (presents)various kinds of information for the operator.

The image processing apparatus 30 is implemented by, for example, apersonal computer or the like, and processes various images. That is,the image processing apparatus 30 incorporates a computer. The computerincludes a main control unit such as a CPU (Central Processing Unit),storage units such as a ROM (Read Only Memory) and RAM (Random AccessMemory), and the like.

In this embodiment, the image processing apparatus 30 includes, as itsfunctional units, an image obtaining unit 31, storage unit 32, imageprocessing unit 33, instruction unit 34, and display control unit 35.Note that the units other than the storage unit 32 are implemented, forexample, when the CPU reads out and executes a program stored in the ROMor the like.

The image processing unit 33 includes a detection unit 41, determinationunit 43, retinal layer analysis unit 44, and alignment unit 47.

The image obtaining unit 31 obtains a tomographic image captured by thetomography apparatus 20 and a fundus image captured by the fundus imagecapturing device 51, and stores these images in the storage unit 32.Note that the storage unit 32 is implemented by, for example, the ROM,RAM, and the like.

The detection unit 41 detects retinal layers from the tomographic imagestored in the storage unit 32.

The retinal layer analysis unit 44 analyzes the retinal layers to beanalyzed. The retinal layer analysis unit 44 includes an analysis unit42, analysis result generation unit 45, and shape data generation unit46.

The determination unit 43 determines whether or not three-dimensionalshape analysis processing of retinal layers is to be executed accordingto an imaging mode (a myopia analysis imaging mode and non-myopiaanalysis imaging mode). Note that the three-dimensional shape analysisindicates processing for generating three-dimensional shape data, andexecuting shape analysis of retinal layers using the shape data.

The analysis unit 42 applies analysis processing to retinal layers to beanalyzed based on the determination result of the determination unit 43.Note that this embodiment will explain a case of macula analysis of amyopia as the three-dimensional shape analysis processing. The analysisresult generation unit 45 generates various data required to present theanalysis result (information indicating states of retinal layers). Theshape data generation unit 46 aligns a plurality of tomographic imagesobtained by imaging, thereby generating three-dimensional shape data.That is, the three-dimensional shape data is generated based on layerinformation of retinal layers.

The alignment unit 47 performs alignment between the analysis result andfundus image, that between fundus images, and the like. The instructionunit 34 instructs information such as imaging parameters according tothe imaging mode set in the tomography apparatus 20.

The example of the arrangement of the image processing system 10 hasbeen described. Note that the functional units arranged in theaforementioned apparatuses need not always be implemented, as shown inFIG. 1, and all or some of these units need only be implemented in anyapparatus in the system. For example, in FIG. 1, the external storagedevice 52, display device 53, and input device 54 are arranged outsidethe image processing apparatus 30. However, these devices may bearranged inside the image processing apparatus 30. Also, for example,the image processing apparatus and tomography apparatus 20 may beintegrated.

An example of a tomographic image capturing screen 60 displayed on thedisplay device 53 shown in FIG. 1 will be described below with referenceto FIG. 2. Note that this screen is displayed when a tomographic imageis to be captured.

The tomographic image capturing screen 60 includes a tomographic imagedisplay field 61, fundus image display field 62, combo box 63 used toset an imaging mode, and capture button 64 used to instruct to capturean image. Note that reference numeral 65 in the fundus image displayfield 62 denotes a mark which indicates an imaging region, and issuperimposed on a fundus image. Reference symbol M denotes a macularregion; D, an optic papilla; and V, a blood vessel.

The combo box 63 allows the user to set, for example, an imaging modefor myopia analysis of (a macular region) or that for non-myopiaanalysis of (the macular region). That is, the combo box 63 has animaging mode selection function. In this case, the imaging mode formyopia analysis is set.

The tomographic image display field 61 displays a tomographic image of afundus. Reference numeral L1 denotes an inner limiting membrane (ILM);L2, a boundary between a nerve fiber layer (NFL) and ganglion cell layer(GCL); and L3, an inner segment outer segment junction (ISOS) of aphotoreceptor cell. Also, reference numeral L4 denotes a pigmentedretinal layer (RPE); and L5, a Bruch's membrane (BM). The aforementioneddetection unit 41 detects any of boundaries of L1 to L5.

An example of the sequence of processing of the image processingapparatus 30 shown in FIG. 1 will be described below with reference toFIGS. 3A and 3B. The sequence of the overall processing at the time ofcapturing of a tomographic image will be described first with referenceto FIG. 3A.

[Step S101]

The image processing apparatus 30 externally obtains a patientidentification number as information required to identify an eye to beexamined. Then, the image processing apparatus 30 obtains informationrelated to the eye to be examined, which information is held by theexternal storage device 52, based on the patient identification number,and stores the obtained information in the storage unit 32.

[Step S102]

The image processing apparatus 30 instructs the image obtaining unit 31to obtain a fundus image from the fundus image capturing device 51 and atomographic image from the tomography apparatus 20 as pre-scan imagesused to confirm an imaging position at the imaging timing.

[Step S103]

The image processing apparatus 30 sets an imaging mode. The imaging modeis set based on a choice of the operator from the combo box 63 used toset the imaging mode, as described in FIG. 2. A case will be describedbelow wherein imaging is to be done in the imaging mode for myopiaanalysis.

[Step S104]

The image processing apparatus 30 instructs the instruction unit 34 toissue an imaging parameter instruction according to the imaging mode setfrom the combo box 63 to the tomography apparatus 20. Thus, thetomography apparatus 20 controls the parameter setting unit 23 to setthe imaging parameters according to the instruction. More specifically,the image processing apparatus 30 instructs to set at least one of theposition of the vision fixation lamp, scanning range, and scanningpattern, and coherence gate position.

The parameter settings in the imaging mode for myopia analysis will bedescribed below. In a parameter setting of the position of the visionfixation lamp, the position of the vision fixation lamp 24 is set to beable to capture an image of the center of a macular region. The imageprocessing apparatus 30 instructs the driving control unit 22 to controlthe light-emitting diodes of the indication unit 24 a according to theimaging parameters. Note that in case of an apparatus with anarrangement which allows to assure a sufficiently broad imaging range,the position of the vision fixation lamp 24 may be controlled to set thecenter between a macular region and optic papilla as that of imaging.The reason why such control is executed is to capture an image of aregion including a macular region so as to execute shape analysis ofretinal layers in a myopia.

In a parameter setting of the scanning range, for example, a range of 9to 15 mm is set as limit values of an imaging range of the apparatus.These values are merely an example, and may be changed as neededaccording to the specifications of the apparatus. Note that the imagingrange is a broader region so as to detect shape change locations withoutany omission.

In a parameter setting of the scanning pattern, for example, a rasterscan or radial scan is set so as to be able to capture athree-dimensional shape of retinal layers.

In a parameter setting of the coherence gate position, the gate positionis set so as to allow imaging based on the EDI method. In a high myopia,a degree of curvature of retinal layers becomes strong, and an image ofretinal layers is unwantedly captured beyond an upper portion side of atomographic image. In this case, retinal layers beyond the upper portionof the tomographic image are folded back and appear in the tomographicimage, and such parameter setting is required to prevent this. Note thatwhen the SS-OCT of the large invasion depth is used as the tomographyapparatus 20, if the position of retinal layers is distant from the gateposition, a satisfactory tomographic image can be obtained. Hence,imaging based on the EDI method is not always done.

[Step S105]

The image processing apparatus 30 instructs the instruction unit 34 toissue an imaging instruction of the eye to be examined to the tomographyapparatus 20. This instruction is issued, for example, when the operatorpresses the capture button 64 of the tomographic image capturing screen60 via the input device 54. In response to this instruction, thetomography apparatus 20 controls the driving controller 22 based on theimaging parameters set by the parameter setting unit 23. Thus, thegalvanometer mirror 21 is activated to capture a tomographic image.

As described above, the galvanometer mirror 21 includes an X-scanner fora horizontal direction, and a Y scanner for a vertical direction. Forthis reason, by changing the directions of these scanners, respectively,a tomographic image can be captured along the horizontal direction (X)and vertical direction (Y) on an apparatus coordinate system. Bysimultaneously changing the directions of these scanners, a scan can bemade in a synthesized direction of the horizontal and verticaldirections. Hence, imaging along an arbitrary direction on a fundusplane can be done. At this time, the image processing apparatus 30instructs the display control unit 35 to display the capturedtomographic image on the display device 53. Thus, the operator canconfirm the imaging result.

[Step S106]

The image processing apparatus 30 instructs the image processing unit 33to detect/analyze retinal layers from the tomographic image stored inthe storage unit 32. That is, the image processing apparatus 30 appliesdetection/analysis processing of retinal layers to the tomographic imagecaptured in the process of step S105.

[Step S107]

The image processing apparatus 30 determines whether or not to endimaging of tomographic images. This determination is made based on aninstruction from the operator via the input device 54. That is, theimage processing apparatus 30 determines whether or not to end imagingof tomographic images based on whether or not the operator inputs an endinstruction.

When the imaging end instruction is input, the image processingapparatus 30 ends this processing. On the other hand, when imaging is tobe continued without ending processing, the image processing apparatus30 executes processes in step S102 and subsequent steps.

Note that when the operator manually modifies the detection result ofretinal layers and the positions of a fundus image and map by processesof steps S206 and S209 (to be described later), the image processingapparatus 30 saves the imaging parameters changed according to suchmodifications in the external storage device 52 upon ending imaging. Atthis time, a confirmation dialog as to whether or not to save thechanged parameters may be displayed to issue an inquiry about whether ornot to change the imaging parameters to the operator.

The detection/analysis processing of retinal layers in step S106 of FIG.3A will be described below with reference to FIG. 3B.

[Step S201]

When the detection/analysis processing of retinal layers is started, theimage processing apparatus 30 instructs the detection unit 41 to detectretinal layers from a tomographic image. This processing will bepractically described below using a tomographic image (display field 61)shown in FIG. 2. In case of a macular region, the detection unit 41applies a median filter and Sobel filter to the tomographic image togenerate images (to be respectively referred to as a median image andSobel image hereinafter). Subsequently, the detection unit 41 generatesprofiles for each A-scan from the generated median image and Sobelimage. A luminance value profile is generated from the median image, anda gradient profile is generated from the Sobel image. Then, thedetection unit 41 detects peaks in the profile generated from the Sobelimage. Finally, the detection unit 41 refers to the profile of themedian image corresponding to portions before and after the detectedpeaks and those between adjacent peaks, thus detecting boundaries ofrespective regions of the retinal layers. That is, L1 (ILM), L2(boundary between the NFL and GCL), L3 (ISOS), L4 (RPE), L5 (BM), andthe like are detected. Note that the following description of thisembodiment will be given under the assumption that an analysis targetlayer is the RPE.

[Step S202]

The image processing unit 30 controls the determination unit 43 todetermine whether or not to execute three-dimensional shape analysis ofthe retinal layer. More specifically, if imaging is done in the imagingmode for myopia analysis, it is determined that the three-dimensionalshape analysis of the retinal layer is to be executed. If imaging isdone without using the myopia analysis mode (in the imaging mode fornon-myopia analysis), it is determined that the three-dimensional shapeanalysis is not to be executed. In this case, analysis based on thedetection result of the detection unit 41 is performed (analysis withoutusing the three-dimensional shape data). Note that even in the imagingmode for myopia analysis, it is determined based on a tomographic imagewhether or not a macular region is included in the tomographic image. Ifno macular region is included in the tomographic image (for example,only an optic papilla is included), it may be determined that(three-dimensional) shape analysis of the retinal layer is not to beexecuted.

The following description of this embodiment will be given under theassumption that a series of processes from imaging to analysis areexecuted in the image processing system 10 which integrates thetomography apparatus 20 and image processing apparatus 30. However, thepresent invention is not limited to this. That is, the image processingapparatus 30 need only execute shape analysis of a retinal layer uponreception of a tomographic image of a macular region using a scanningpattern required to obtain a three-dimensional shape, and suchintegrated system need not always be adopted. For this reason, the imageprocessing apparatus 30 can execute shape analysis of a retinal layerfor a tomographic image captured by an apparatus other than thetomography apparatus 20 based on information at the time of imaging.However, when shape analysis is not required, the shape analysisprocessing may be skipped.

[Step S203]

The image processing apparatus 30 instructs the shape data generationunit 46 to generate three-dimensional shape data. The three-dimensionalshape data is generated to execute shape analysis based on the detectionresult of the retinal layer in the process of step S201.

In this case, when the scanning pattern at the time of imaging is, forexample, a raster scan, a plurality of adjacent tomographic images arealigned. In alignment of tomographic images, for example, an evaluationfunction which represents a similarity between two tomographic images isdefined in advance, and tomographic images are deformed to maximize thisevaluation function value.

As the evaluation function, for example, a method of evaluating pixelvalues (for example, a method of making evaluation using correlationcoefficients) may be used. As the deformation processing of images,processing for making translation and rotation using affinetransformation may be used. After completion of the alignment processingof the plurality of tomographic images, the shape data generation unit46 generates three-dimensional shape data of a layer as a shape analysistarget. The three-dimensional shape data can be generated by preparing,for example, 512×512×500 voxel data, and assigning labels to positionscorresponding to coordinate values of layer data of the detected retinallayer.

On the other hand, when the scanning pattern at the time of imaging is aradial scan, the shape data generation unit 46 aligns tomographicimages, and then generates three-dimensional shape data in the samemanner as described above. However, in this case, alignment in a depthdirection (Z direction of the tomographic image (display field 61) shownin FIG. 2) is made using only region information near the centers ofadjacent tomographic images. This is because in case of the radial scan,even adjacent tomographic images include coarse information at two endscompared to the vicinity of each center thereof, shape changes arelarge, and such information is not used as alignment information. As thealignment method, the aforementioned method can be used. Aftercompletion of the alignment processing, the shape data generation unit46 generates three-dimensional shape data of a layer as a shape analysistarget.

In case of the radial scan, for example, 512×512×500 voxel data areprepared, and layer data as a shape analysis target of respectivetomographic images are evenly circularly rotated and expanded. Afterthat, in the expanded layer data, interpolation processing is executedbetween adjacent shape data in the circumferential direction. With theinterpolation processing, shape data at non-captured positions aregenerated. As the interpolation processing method, processing such aslinear interpolation or nonlinear interpolation may be applied. Thethree-dimensional shape data can be generated by assigning labels topositions corresponding to coordinate values obtained by interpolatingbetween layer data of the detected retinal layer.

Note that numerical values of the voxel data described above are merelyan example, and can be changed as needed depending on the number ofA-scans at the time of imaging and the memory size of the apparatuswhich executes the processing. Since large voxel data have a highresolution, they can accurately express shape data. However, such voxeldata suffers a low execution speed, and have a large memory consumptionamount. On the other hand, although small voxel data have a lowresolution, they can assure a high execution speed and have a smallmemory consumption amount.

[Step S204]

The image processing apparatus 30 instructs the analysis unit 42 toexecute the three-dimensional shape analysis of the retinal layer. Asthe shape analysis method, a method of measuring an area and volume ofthe retinal layer will be exemplified below.

This processing will be described below with reference to FIG. 4. FIG. 4illustrates three-dimensional shape data (RPE), measurement surface(MS), area (Area), and volume (Volume).

An area measurement will be described first. In the three-dimensionalshape data of the RPE generated in the process of step S203, a flat(planar) measurement surface (MS) is prepared at a place of layer datalocated at the deepest portion in the Z direction (optical axisdirection). Then, the measurement surface (MS) is moved at givenintervals in a shallow direction (an origin direction of the Z-axis)from there. When the measurement surface (MS) is moved from the deepportion of the layer in the shallow direction, it traverses a boundaryline of the RPE.

An area (Area) is obtained by measuring an internal planar regionbounded by the measurement surface (MS) and the boundary line with theRPE. More specifically, the area (Area) is obtained by measuring an areaof an intersection region between the measurement surface (MS) andboundary line with the RPE. In this manner, the area (Area) is across-sectional area of the three-dimensional retinal layer shape data.Upon measuring a cross-sectional area at a position of a referenceportion, when a curvature of the retinal layer is strong, a small areais obtained; when the curvature of the retinal layer is moderate, alarge area is obtained.

A volume (Volume) can be obtained by measuring a whole internal regionbounded by the measurement surface (MS) and the boundary line with theRPE using the measurement surface (MS) used in the measurement of thearea (Area). Upon measuring a volume in a downward direction along the Zdirection from a position of a reference position, when a curvature ofthe retinal layer is strong, a large volume is obtained; when thecurvature of the retinal layer is moderate, a small volume is obtained.In this case, the reference position can be set at an Bruch's membraneopening position with reference to a portion. Alternatively, a givenheight such as 100 μm or 500 μm from the deepest position of the RPE maybe used as a reference. Note that when the number of voxels included ina region to be measured is counted upon measuring the area or volume,the area or volume is calculated by multiplying the number of voxels bya physical size per voxel.

[Step S205]

After the shape analysis, the image processing apparatus 30 instructsthe analysis result generation unit 45 to generate an analysis result(for example, a map, graph, or numerical value information) based on thethree-dimensional shape analysis result.

A case will be described below wherein a contour line map is generatedas the three-dimensional shape analysis result. The contour line map isused when the measurement results of the area and volume are to bedisplayed.

FIG. 5A shows an example of a contour line map 71. The contour line map71 is an overall contour line map. Reference numeral 72 denotes contourlines drawn at given intervals; and 73, a portion located at the deepestportion in the Z direction of the three-dimensional retinal layer shapedata.

When the contour line map 71 is to be generated, a lookup table for acontour line map is prepared, and the map is color-coded according tothe volumes with reference to the table. In this case, the lookup tablefor the contour line map may be prepared according to the volumes. Thus,the operator can recognize changes of the shape and volume at a glanceon the map.

More specifically, the contour lines 72 are drawn at given intervals tohave the portion 73 in FIG. 5A as a bottom, and the colors of the mapare set according to the volumes. For this reason, when the height(depth) of the measurement surface is changed from the portion locatedat the deepest position in the Z direction, the operator can recognizehow to increase a volume. More specifically, when it is set tocolor-code 1 mm³ as blue and 2 mm³ as yellow, the operator can recognizethe relationship between the shape and volume by checking whether bluecorresponds to the height (depth) of 100 μm or 300 μm of the measurementsurface from the portion located at the deepest position in the Zdirection. Therefore, the operator can recognize the overall volume uponmeasuring the shape of the retinal layer to be measured by confirming acolor of an outermost contour of the map.

Also, the operator can confirm a volume value corresponding to a height(depth) by confirming a color near each internal contour line. Note thatthe lookup table used upon setting colors of the contour line map may beprepared according to areas in place of the volumes. Alternatively, thelookup table may be prepared according to heights (depths) to theportion 73 located at the deepest position in the Z direction. Althoughnot shown, numerical values may be displayed together on respectivecontour lines so as to allow the operator to understand the heights(depths) of the contour lines. As an interval of a distance whichexpresses a contour line, for example, a 100-μm interval along the Zdirection is set. Note that the contour line map may be either a coloror grayscale map, but visibility is high in case of the color map.

An outer shape size of the contour line map changes depending on theheight (depth) from the portion located at the deepest position in the Zdirection to the measurement surface (MS). FIG. 5B shows an example of acontour line map 74 when the height (depth) of the measurement surface(MS) is changed.

A case in which a curvature of the retinal layer is to be measured asthe three-dimensional shape analysis will be described below using thetomographic image (display field 61) shown in FIG. 2. In the followingdescription, a case will be explained wherein the abscissa is defined asan x-coordinate axis, the ordinate is defined as a z-coordinate axis,and a curvature of a boundary line of the layer (RPE) as an analysistarget is calculated. A curvature κ can be obtained by calculating, atrespective points of the boundary line:

$\begin{matrix}{\kappa = \frac{\frac{\mathbb{d}^{2}z}{\mathbb{d}x^{2}}}{\left( {1 + \left( \frac{\mathbb{d}z}{\mathbb{d}x} \right)} \right)^{\frac{3}{2}}}} & (1)\end{matrix}$The sign of the curvature κ reveals that the shape is upward or downwardconvex, and the magnitude of a numeral value reveals a curved degree ofthe shape. For this reason, if upward convex is expressed by “+” anddownward convex is expressed by “−”, if each tomographic image includesa − region, + region, and − region as the signs of the curvature, thelayer has a W-shape.

Note that the case has been explained wherein the curvature of theboundary line of the tomographic image is calculated in this case.However, the present invention is not limited to such specific curvaturecalculation, and three-dimensional curvatures may be calculated from thethree-dimensional shape data. In this case, after the shape analysis,the image processing apparatus 30 instructs the analysis resultgeneration unit 45 to generate a curvature map based on the analysisresult.

FIG. 5C shows an example of the curvature map. In this case, a portionhaving a strong curvature is expressed by a dark color, and that havinga moderate curvature is expressed by a light color. More specifically,the color density is changed depending on the curvatures. Note thatcolors to be set in the curvature map may be changed depending onpositive and negative curvature values with reference to a curvaturevalue=0. Thus, the operator can recognize whether or not the retinashape is smooth and whether it is an upward or downward convex shape bychecking the map.

[Step S206]

The image processing apparatus 30 instructs the display control unit 35to display a tomographic image, the detection result of the layer (RPE)detected by the detection unit 41, and various shape analysis results(map, graph, and numerical value information) generated by the analysisresult generation unit 45 on the display device 53.

FIG. 6 shows an example of a tomographic image observation screen 80displayed on the display device 53 shown in FIG. 1. This screen isdisplayed after completion of the analysis of tomographic images (thatis, it is displayed by the process of step S206).

The tomographic image observation screen 80 includes a tomographic imagedisplay section 91 including a tomographic image display field 81, and afundus image display section 94 including a fundus image display field82. The tomographic image observation screen 80 also includes a firstanalysis result display section 96 including a first analysis result 84,and a second analysis result display section 98 including secondanalysis results 85 and 86.

Details of the tomographic image display section 91 including thetomographic image display field 81 will be described first. Thetomographic image display field 81 displays segmentation results (L1 toL5) obtained by detecting the respective layers of the retinal layersand the measurement surface (MS) which are superimposed on the capturedtomographic image. The tomographic image display field 81 highlights thesegmentation result of the retinal layer (RPE (L4) in this embodiment)as an analysis target.

On the tomographic image display field 81, a hatched region 81 a boundedby the measurement surface (MS) and the retinal layer (RPE (L4)) as ananalysis target is a measurement target region of the area and volume.At this time, on the hatched region 81 a, a color according to thevolume measurement result is displayed to have a predeterminedtransparency α. The same color to be set as that in the lookup table forthe contour line map can be used. The transparency α is, for example,0.5.

A combo box 92 is provided to allow the operator to select whether thetomographic image is displayed at an OCT ratio or 1:1 ratio. In thiscase, the OCT ratio is that expressed by a resolution in the horizontaldirection (X direction) and that in the vertical direction (Ydirection), which are obtained based on the number of A-scans at thetime of imaging. The 1:1 ratio is that which adjusts a physical size perpixel in the horizontal direction and that per pixel in the verticaldirection, which are obtained based on the number of A-scans used tocapture a given range (mm).

A combo box 93 is provided to allow the operator to switch atwo-dimensional (2D)/three-dimensional (3D) display mode. In the 2Ddisplay mode, one slice of the tomographic image is displayed; in the 3Ddisplay mode, the three-dimensional shape of the retinal layersgenerated from the boundary line data of the retinal layers isdisplayed.

More specifically, when the operator selects the 3D display mode at thecombo box 93, a tomographic image shown in one of FIGS. 7A to 7C isdisplayed in the tomographic image display field 81.

FIG. 7A shows a mode when the RPE is displayed at the OCT ratio in the3D display mode. In this case, the measurement surface (MS) issimultaneously displayed in the 3D display mode.

In this case, check boxes 101 to 104 corresponding to the respectivelayers of the retina are displayed below the tomographic image displayfield 81. More specifically, the check boxes corresponding to the ILM,RPE, BM, and MS are displayed, and the operator can switchdisplay/non-display states of the respective layers using these checkboxes.

When the measurement surface (MS) is expressed by a plane, itstransparency α assumes a value which is larger than 0 and is smallerthan 1. In a state in which the transparency is 1, and when the retinallayer shape and measurement surface are overlaid, the shape isunwantedly covered, and the three-dimensional shape of the retinallayers cannot be recognized from the upper side. Alternatively, themeasurement surface (MS) may be expressed by a grid pattern in place ofthe plane. In case of the grid pattern, the transparency α of themeasurement surface (MS) may be set to be 1. As for a color of themeasurement surface (MS), a color according to a measurement value (areaor volume) at the location of the measurement surface (MS) need only beselected with reference to the lookup table for the contour line map.

In this case, the operator can move the position of the measurementsurface (MS) via the input device 54. For this reason, when the positionof the measurement surface (MS) is changed, the image processingapparatus 30 changes the contour line map shape in synchronism with thatchange, as described in FIGS. 5A and 5B.

A text box 105 is an item used to designate a numerical value. Theoperator inputs, to the text box 105, a numerical value indicating aheight (depth) of the measurement surface (MS) from the portion at thedeepest position in the Z direction via the input device 54. Forexample, when the operator inputs a numerical value such as 100 μm or300 μm, the measurement surface (MS) is moved to that position, and thecontour line map is changed accordingly. Thus, the operator cansimultaneously recognize the position in the three-dimensional shape andthe contour line map at that time, and can also recognize a volume valueand area value at that time.

As another example, the operator may input a volume value in the textbox 105. In this case, when the operator inputs a numerical value suchas 1 mm³ or 2 mm³, he or she can simultaneously recognize the positionin the three-dimensional shape and the contour line map at that time,which correspond to that volume. Furthermore, the operator can alsorecognize a height (depth) of the measurement surface (MS) from theportion at the deepest position in the Z direction at that time.

Subsequently, FIG. 7B shows a display mode when the measurement surface(MS) is set in a non-display state. Other display items are the same asthose in FIG. 7A. In case of FIG. 7B, the check box 102 is selected todisplay the RPE alone. In this manner, only the three-dimensional shapeof a fundus can be displayed.

FIG. 7B shows the 3D display mode of the RPE at the OCT ratio, whileFIG. 7C shows a mode when the RPE is displayed at the 1:1 ratio in the3D display mode. That is, the 1:1 ratio is selected at the combo box 92.

Note that since this embodiment has explained the RPE as the analysistarget, when the 2D/3D display mode is switched at the combo box 93, theRPE shape is displayed in the 2D/3D display mode. However, the presentinvention is not limited to this. For example, when the Bruch's membrane(BM) is selected as an analysis target, the Bruch's membrane (BM) isdisplayed in the 2D/3D display mode.

In this case, the position measured by the measurement surface may beschematically displayed on the tomographic image display field 81. Adisplay mode in this case will be described below with reference toFIGS. 8A to 8D.

FIG. 8A shows a mode in which an object 110 indicating the positionmeasured by the measurement surface in the currently displayedtomographic image is superimposed on the tomographic image. Referencesymbol MS′ denotes a measurement surface (schematic measurement surface)on the object 110. The three-dimensional shape of the retinal layer isrotated in the upper, lower, right, and left directions by aninstruction input by the operator via the input device 54. For thisreason, in order to allow the operator to recognize the positionalrelationship between the measurement surface (MS) and retinal layer, theobject 110 and schematic measurement surface MS′ present an index of thepositional relationship.

Note that when the operator changes the positional relationship betweenthe object 110 and schematic measurement surface MS′ via the inputdevice 54, the displayed three-dimensional shape of the retinal layer isalso changed in synchronism with that change. In this case, since theregion of the retinal layer as the analysis target is also changed, thefirst analysis result 84 and second analysis results 85 and 86 arechanged in synchronism with that change.

Some display modes of the object 110 will be exemplified below. Forexample, as shown in FIGS. 8B and 8C, tomographic images correspondingto respective section positions may be displayed. In this case, in theobject 110, tomographic images at vertical and horizontal positions,which intersect the central position in consideration of athree-dimensional shape, are displayed. Alternatively, as shown in FIG.8D, an abbreviation such as “S” or “I” indicating “superior” or“inferior” may be displayed.

Details of the fundus image display section 94 including the fundusimage display field 82 shown in FIG. 6 will be described below. On Thefundus image display field 82, an imaging position and its scanningpattern mark 83 are superimposed on the fundus image. The fundus imagedisplay section 94 is provided with a combo box 95 which allows theoperator to switch a display format of the fundus image. In this case,an SLO image is displayed as the fundus image.

A case will be described below with reference to FIGS. 9A and 9B whereinthe operator switches the display format of the fundus image from thecombo box 95. In this case, a case in which “SLO image+map” aresimultaneously displayed and a case in which “fundus photo (secondfundus image)+SLO image (first fundus image)+map” are simultaneouslydisplayed as the display formats of the fundus image will beexemplified. Note that the SLO image (first fundus image) can be atwo-dimensional fundus image captured simultaneously with a tomographicimage, and for example, it may be an integrated image generated byintegrating tomographic images in the depth direction. The fundus photo(second fundus image) may be a two-dimensional fundus image captured ata timing different from a tomographic image, and for example, a contrastradiographic image or the like may be used.

FIG. 9A shows a display mode when the operator selects “SLO image+map”from the combo box 95. Reference numeral 201 denotes an SLO image; and200, a map. The SLO image 201 and map 200 are aligned by the alignmentunit 47 described using FIG. 1. The SLO image and map are aligned bysetting the position and size of the map based on the position of thevision fixation lamp and the scanning range at the time of imaging.

When a superimposing result of the map 200 on the SLO image 201 isdisplayed as the fundus image, the transparency α of the SLO image 201to be displayed is set to be 1, and that of the map 200 to be displayedis set to be smaller than 1 (for example, 0.5). These parameters of thetransparencies α are those to be set when the operator selects targetdata for the first time. The operator can change these parameters viathe input device 54 as needed. The parameters changed by the operatorare stored in, for example, the external storage device 52. When thesame target data is opened for the next or subsequent time, displayprocessing is performed according to the parameters previously set bythe operator.

Subsequently, FIG. 9B shows a display mode when the operator selects“fundus photo+SLO image+map” from the combo box 95. Reference numeral202 denotes a fundus photo (second fundus image).

In this case, in order to superimpose the map 200 on the fundus photo202 (that is, to superimpose the map 200 on the second fundus image),the SLO image 201 is used. This is because the fundus photo 202 iscaptured at a timing different from a tomographic image, and the imagingposition and range cannot be recognized based on the map 200 alone.Hence, using the SLO image 201, the fundus photo 202 and map 200 can bealigned. The fundus photo 202 and SLO image 201 are aligned by thealignment unit 47 described using FIG. 1.

As the alignment method, for example, a blood vessel feature may beused. As a detection method of blood vessels, since each blood vesselhas a thin linear structure, blood vessels are extracted using a filterused to emphasize the linear structure. As the filter used to emphasizethe linear structure, when a line segment is defined as a structuralelement, a filter which calculates a difference between an average valueof image density values in the structural element and that in a localregion which surrounds the structural element may be used. Of course,the present invention is not limited to such specific filter, and adifference filter such as a Sobel filter may be used. Alternatively,eigenvalues of a Hessian matrix may be calculated for each pixel of adensity value image, and a line segment-like region may be extractedbased on combinations of two eigenvalues obtained as calculationresults. The alignment unit 47 aligns the fundus photo 202 and SLO image201 using blood vessel position information detected by these methods.

Since the SLO image 201 and map 200 can be aligned by the aforementionedmethod, the fundus photo 202 and map 200 can also be consequentlyaligned. When a superimposing result of the map 200 on the fundus photo202 is displayed on the display field 82 of the fundus image displaysection 94, the transparency α of the fundus photo 202 to be displayedis set to be 1. Also, the transparency of the SLO image 201 to bedisplayed and that of the map 200 to be displayed are set to be smallerthan 1 (for example, 0.5). Of course, the values of the transparencies αof the SLO image 201 and map 200 need not always be the same. Forexample, the value of the transparency α of the SLO image 201 may be setto be 0.

Note that when an eye to be examined is a diseased eye, the alignmentprocessing by the alignment unit 47 may often fail. An alignment failureis determined when a maximum similarity does not become equal to orlarger than a threshold upon calculation of an inter-image similarity.Even when the maximum similarity becomes equal to or larger than thethreshold, the end of alignment processing at an anatomically abnormalposition determines a failure. In this case, the SLO image 201 and map200 need only be displayed at a position (for example, the center of animage) and initial transparencies of initial parameters, which are setin advance. A failure message of the alignment processing is displayed,thus prompting the operator to execute position correction processingvia the input device 54.

In this case, when the operator modifies the position and changesparameters of the transparencies α, if he or she moves,enlarges/reduces, and rotates the SLO image 201, the map 200 on the SLOimage 201 is simultaneously moved, enlarged/reduced, and rotated. Thatis, the SLO image 201 and map 200 operate as a single image. However,the transparencies α of the SLO image 201 and map 200 are independentlyset. The parameters which are changed by the operator via the inputdevice 54 are stored in the external storage device 52, and the next orsubsequent display operation is made according to the set parameters.

In this manner, the fundus image display field 82 in the fundus imagedisplay section 94 displays the two-dimensional fundus image, the mapsuperimposed on the fundus image, and the like. Note that FIGS. 9A and9B have explained the case in which the contour line map is superimposedin association with a corresponding position on the fundus image.However, the present invention is not limited to this. That is, acurvature map, layer thickness map, and the like may be displayed.

When an analysis target layer is to be changed, the operator can select,for example, the segmentation results (L1 to L5) from the tomographicimage on the display field 61 shown in FIG. 2 via the input device 54.When an analysis target layer is switched, the image processingapparatus 30 instructs the display control unit 35 to normally displaythe segmentation result layer highlighted so far, and to highlight a newanalysis target layer. Thus, the analysis results of an arbitrary layercan be displayed.

Details of the first analysis result display section 96 including thefirst analysis result 84 shown in FIG. 6 will be described below.

The first analysis result 84 displays a shape analysis map generated bythe analysis result generation unit 45. A combo box 97 allows theoperator to select a map type of the first analysis result 84. In thiscase, the shape analysis map indicated by the first analysis result 84is a contour line map.

The type of the shape analysis map indicated as the first analysisresult 84 and that of the shape analysis map superimposed on the fundusimage described using FIGS. 9A and 9B can be changed in synchronism witheach other by designating the type from the combo box 97. Furthermore,the displayed contents on the second analysis result display section 98to be described later are also changed in synchronism with suchdesignation.

An example of the tomographic image observation screen 80 when thecurvature result is displayed as the analysis result will be describedbelow with reference to FIG. 10.

When the analysis result to be displayed is switched from that of thearea and volume to the curvature analysis result, results displayed onthe tomographic image display section 91, fundus image display section94, first analysis result display section 96, and second analysis resultdisplay section 98 have contents shown in FIG. 10.

More specifically, on the tomographic image display field 81, thesegmentation results (L1 to L5) of the respective detected layers of theretinal layers are superimposed on a captured tomographic image. Acurvature map is displayed as the first analysis result 84, and acurvature graph is displayed as the second analysis result 88. On thefundus image display field 82, an image obtained by superimposing theSLO image (first fundus image) 201, the fundus photo (second fundusimage) 202, and a curvature map 203 is displayed.

Details of the second analysis result display section 98 including thesecond analysis results 85 and 86 shown in FIG. 6 will be describedbelow.

As the second analysis result 85, a shape analysis graph generated bythe analysis result generation unit 45 is displayed. In this case, agraph obtained by measuring the area and volume is displayed. Theabscissa plots the height (depth), and the ordinate plots the volume. Asolid curve 87 represents the volume.

As the second analysis result 86, a shape analysis result is displayedas a table. The table displays an area and volume at a height (forexample, 100 μm, 500 μm, or the like) of a given reference value, and anarea and volume corresponding to a height (a position of the Bruch'smembrane opening) when a certain portion is used as a reference.

A case will be described below with reference to FIG. 11 wherein areaand volume results are displayed on one graph (second analysis result85). In this case, the abscissa plots the height (depth), the ordinateon the left side of the graph plots the volume, and that on the rightside of the graph plots the area. A broken curve 89 is a graphindicating the area, and the solid curve 87 is a graph indicating thevolume.

[Step S207]

The image processing apparatus 30 instructs the analysis unit 42 toexecute shape analysis based on the detection result of the retinallayer. In this analysis processing, analysis using the detection resultof the retinal layer is executed without generating three-dimensionalshape data, and the like. For example, the analysis of a layer thicknessor the like is executed.

[Step S208]

The image processing apparatus 30 instructs the analysis resultgeneration unit 45 to generate analysis results (for example, a map,graph, and numerical value information) based on the analysis result.

[Step S209]

The image processing apparatus 30 displays a tomographic image, thedetection results of layers (RPE, ILM, and the like) detected by thedetection unit 41, and analysis results (map, graph, and numerical valueinformation) generated by the analysis result generation unit 45 on thedisplay device 53.

FIG. 12 shows an example of the tomographic image observation screen 80displayed on the display device 53 by the process of step S209.

In this case, a layer thickness map is displayed as a first analysisresult 302, and a layer thickness graph is displayed as a secondanalysis result 301. That is, the analysis results using the detectionresult of the retinal layer are displayed as the first and secondanalysis results 302 and 301.

In this case as well, as in the process of step S206, parameters changedby the operator via the input device 54 are stored in the externalstorage device 52, and the next or subsequent display operation is madeaccording to the set parameters.

As described above, according to the first embodiment, an indexindicating the region of the retinal layer as the three-dimensionalshape analysis target can be presented (displayed) together with thethree-dimensional shape data of the retinal layer detected from thetomographic image of the eye to be examined.

Thus, the operator can effectively recognize the correspondencerelationship between the three-dimensional shape of the retinal layerand its analysis target region.

Second Embodiment

The second embodiment will be described below. The second embodimentwill explain a case in which a tomographic image is displayedsimultaneously in 2D and 3D display modes. More specifically, the secondembodiment will explain a case in which a tomographic image,three-dimensional shape data, and analysis results are displayed side byside. Note that the second embodiment will exemplify, as a tomographicimage, that captured in a radial scan.

An example of a tomographic image observation screen 400 according tothe second embodiment will be described below with reference to FIG. 13.

The tomographic image observation screen 400 includes a firsttomographic image display section 410 including a two-dimensionaltomographic image display field 411, and a second tomographic imagedisplay section 430 including a three-dimensional tomographic imagedisplay field 431. Furthermore, the tomographic image observation screen400 includes a first analysis result display section 440 including afirst analysis result 442, and a second analysis result display section420 including second analysis results 421 and 422.

In this embodiment, since the first analysis result display section 440and second analysis result display section 420 are the same as thoseshown in FIGS. 6 and 9A used to explain the aforementioned firstembodiment, a description thereof will not be repeated. Also, since thesecond tomographic image display section 430 is the same as that shownin FIGS. 7A to 7C used to explain the first embodiment, a descriptionthereof will not be repeated. In this embodiment, the first tomographicimage display section 410 will be mainly described.

The first tomographic image display section 410 includes a slider bar412 used to change a viewpoint position (slice position) of atomographic image, and an area 413 used to display a slice number of atomographic image in addition to the two-dimensional tomographic imagedisplay field 411.

The relationship between a viewpoint direction 503 in athree-dimensional shape and the tomographic image (display field 411)shown in FIG. 13 will be described below with reference to FIG. 14.Reference numeral 500 denotes an overview of a fundus when thethree-dimensional shape of the tomographic image is viewed from theabove in a depth direction (Z direction). Radial lines 501 indicateimaging slice positions of tomographic images. A broken line 502indicates a slice position corresponding to the currently displayedtomographic image (display field 411) of the imaging slice positions501, and this slice position is orthogonal to the viewpoint direction503. More specifically, the tomographic image at the position 502 isthat displayed on the tomographic image display field 411.

A case will be described below wherein the operator operates thetomographic image observation screen 400 described using FIG. 13 via aninput device 54.

Assume that the operator moves the position of the slider bar 412 fromthe center to one end via the input device 54. Then, in synchronism withthat operation, an image processing apparatus 30 changes the viewpointposition of the three-dimensional tomographic image (three-dimensionalshape data) currently displayed on the display field 431. Also, theimage processing apparatus 30 changes the slice position of thetwo-dimensional tomographic image currently displayed on the displayfield 411 to a viewpoint position corresponding to a tomographic imagedesignated by the slider bar 412.

Note that when the operator operates the slider bar 412, the position ofthat bar is continuously changed in place of being discretely changed.For this reason, the position of the three-dimensional shape data of thecurrently displayed tomographic image is also continuously changed. Morespecifically, the image processing apparatus 30 displays so that thethree-dimensional shape data of the currently displayed tomographicimage is rotated with reference to the center in the vertical direction.

FIG. 15 shows an example of a screen display of the two-dimensionaltomographic image on the display field 411 and the three-dimensionaltomographic image (three-dimensional shape data) on the display field431 when the position of the slider bar 412 is moved to one end. In thiscase, the relationship between a viewpoint direction 507 in thethree-dimensional shape and the two-dimensional tomographic image(display field 411) becomes that shown in FIG. 16.

When the position of the slider bar 412 is operated, the imageprocessing apparatus 30 changes the slice position of thetwo-dimensional tomographic image currently displayed on the displayfield 411, and also changes the viewpoint position of thethree-dimensional tomographic image (three-dimensional shape data)corresponding to that slice position. Note that the viewpoint positionof the three-dimensional shape data can be changed according to theoperation of the slider bar 412 used to change the slice position of thetwo-dimensional tomographic image, and vice versa. That is, when theoperator changes the viewpoint position of the three-dimensional shapedata via the input device 54, the slice position of the two-dimensionaltomographic image and the position of the slider bar 412 may be changedaccordingly.

As described above, according to the second embodiment, thetwo-dimensional tomographic image and three-dimensional tomographicimage are simultaneously displayed, and their display modes can bechanged in synchronism with each other in response to an operation bythe operator.

The representative embodiments of the present invention have beendescribed. However, the present invention is not limited to the aboveand illustrated embodiments, and the present invention can be practicedwhile being modified as needed without departing from its scope.

For example, on some screens described in the aforementioned first andsecond embodiments, check boxes, combo boxes, and the like are arranged.Radio buttons, list boxes, buttons, and the like may be changed asneeded as usage. For example, components described as the combo boxes inthe aforementioned description may be implemented as list boxes.

Other Embodiments

Aspects of the present invention can also be realized by a computer of asystem or apparatus (or devices such as a CPU or MPU) that reads out andexecutes a program recorded on a memory device to perform the functionsof the above-described embodiment(s), and by a method, the steps ofwhich are performed by a computer of a system or apparatus by, forexample, reading out and executing a program recorded on a memory deviceto perform the functions of the above-described embodiment(s). For thispurpose, the program is provided to the computer for example via anetwork or from a recording medium of various types serving as thememory device (for example, computer-readable medium).

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

This application claims the benefit of Japanese Patent Application No.2012-015933, filed Jan. 27, 2012, which is hereby incorporated byreference herein in its entirety.

What is claimed is:
 1. An image processing system comprising: anobtaining unit configured to obtain a plurality of tomographic images ofan eye to be examined; an extraction unit configured to extract aretinal layer from the plurality of tomographic images; a generationunit configured to generate three-dimensional shape data of the retinallayer based on the retinal layer extracted by the extraction unit; asetting unit configured to set a flat surface that is movable in a depthdirection of the retinal layer extracted by the extraction unit at areference position of the three-dimensional shape data of the retinallayer; a measurement unit configured to measure at least one of an areaand a volume of the retinal layer, based on the three-dimensional shapedata of the retinal layer and the flat surface at the referenceposition, wherein the area is an area of the flat surface surrounded bythe three-dimensional shape data of the retinal layer, and the volume isa volume of a region surrounded by the three-dimensional shape data ofthe retinal layer and the flat surface; and a display control unitconfigured to display at least one of the area and the volume on adisplay device.
 2. The system according to claim 1, further comprising aselection unit configured to select a predetermined layer from retinallayers, wherein the generation unit generates the three-dimensionalshape data of the selected layer.
 3. The system according to claim 1,further comprising an imaging mode selection unit configured to selectan imaging mode from a plurality of imaging modes used to capture animage of an eye to be examined, wherein at least one of a position of avision fixation lamp, a scanning range of irradiating light, a scanningpattern of the irradiating light, and an imaging position along anoptical axis direction of the irradiating light is changed according tothe imaging mode selected by the imaging mode selection unit.
 4. Thesystem according to claim 1, wherein the display control unit displaysan index indicating a positional relationship between thethree-dimensional shape data of the retinal layer and the flat surfaceon the display device.
 5. The system according to claim 4, wherein theindex is configured to include an object indicating thethree-dimensional shape data of the retinal layer and a schematic flatsurface indicating the flat surface.
 6. The system according to claim 5,wherein the display control unit displays the three-dimensional shapedata of the retinal layer, the index, and at least one of the area andthe volume on the display device.
 7. The system according to claim 6,wherein when an operator changes the positional relationship between theobject and the schematic flat surface displayed on the display devicevia an input device, the measurement unit applies the three-dimensionalshape analysis to a region changed accordingly, and the display controlunit changes and displays the result of the three-dimensional shapeanalysis in synchronism with the change of the positional relationshipbetween the object and the schematic flat surface.
 8. The systemaccording to claim 1, wherein the display control unit displays atwo-dimensional tomographic image, which refers to retinal layers from apredetermined viewpoint position, and three-dimensional shape data ofthe retinal layer side by side on the display device, and when anoperator changes the viewpoint position via an input device, the displaycontrol unit changes and displays the viewpoint position of thetwo-dimensional tomographic image and the three-dimensional shape dataof the retinal layer in synchronism with that change.
 9. A processingmethod of an image processing system, comprising: obtaining a pluralityof tomographic images of an eye to be examined; extracting a retinallayer from the plurality of tomographic images; generatingthree-dimensional shape data of the retinal layer based on the extractedretinal layer; setting a flat surface that is movable in a depthdirection of the extracted retinal layer at a reference position of thethree-dimensional shape data of the retinal layer; measuring at leastone of an area and a volume of the retinal layer, based on thethree-dimensional shape data of the retinal layer and the flat surfaceat the reference position, wherein the area is an area of the flatsurface surrounded by the three-dimensional shape data of the retinallayer, and the volume is a volume of a region surrounded by thethree-dimensional shape data of the retinal layer and the flat surface;and displaying at least one of the area and the volume on a displaydevice.
 10. A non-transitory computer readable storage medium storing aprogram for controlling a computer to execute the processing method ofclaim
 9. 11. An image processing system comprising: an obtaining unitconfigured to obtain a plurality of tomographic images of an eye to beexamined; an extraction unit configured to extract a retinal layer fromthe plurality of tomographic images; a generation unit configured togenerate three-dimensional shape data of the retinal layer based on theretinal layer extracted by the extraction unit; a setting unitconfigured to set a flat surface that is movable in a depth direction ofthe retinal layer extracted by the extraction unit at a referenceposition of the three-dimensional shape data of the retinal layer; ameasurement unit configured to measure an intersection region betweenthe three-dimensional shape data of the retinal layer and the flatsurface at the reference position; and a display control unit configuredto display the measured intersection region on a display device, whereinthe measurement unit is configured to measure an area of theintersection region surrounded by the three-dimensional shape data ofthe retinal layer and a volume of a region surrounded by theintersection region and the three-dimensional shape data of the retinallayer, and the display control unit is configured to display a graphindicating at least one of the area and the volume.
 12. The systemaccording to claim 11, wherein the measurement unit measures a pluralityof intersection regions between the three-dimensional shape data of theretinal layer and the flat surface, in a case where the flat surface ismoved at constant intervals in the depth direction.
 13. The systemaccording to claim 12, wherein the display control unit displays atomographic image on which a line indicating a position of the flatsurface is superimposed.
 14. The system according to claim 1, whereinthe measurement unit measures an area and a volume with respect to eachflat surface corresponding to each of a plurality of different positionslocated in a depth direction of the three-dimensional shape data of theretinal layer, and obtains a plurality of areas and volumescorresponding to respective flat surfaces, and the display control unitdisplays the obtained plurality of areas and volumes on the displaydevice.
 15. The system according to claim 1, wherein the area and thevolume are values used for obtaining a curvature of the retinal layer.